1. Field of the Invention
The present invention relates to a digital camera. In particular, the present invention discloses a digital camera that has low memory requirements for a frame buffer.
2. Description of the Prior Art
Enormous advances in digital electronics over the past year have resulted in more and more consumers switching over from traditionally analog devices to their digital equivalents. A particularly relevant example of this is the digital camera, which is rapidly replacing film-based cameras. Digital cameras enable a user to preview a frame, and to select those frames that are to be kept, while discarding unwanted frames. This offers a great advantage over the previous incarnation of film-based cameras, in which the frames were inerasably caught within the emulsion of the film, and which could not be viewed until the film was developed often many weeks or months after the frame itself was taken.
Please refer to FIG. 1 and FIG. 2. FIG. 1 is a flow chart for a prior art digital camera. FIG. 2 is a partial block diagram of an optical sensor 20. The prior art digital camera utilizes an image sensor system 2 to capture a single image frame. The image sensor system 2 is itself quite complex, including many components, such as lenses, an aperture diaphragm, etc. In particular, though, at the heart of the image sensor system 2 lies the optical sensor 20. The optical sensor 20 is typically a CCD or CMOS device, and serves as the “film” for the digital camera. The optical sensor 20 comprises a plurality of pixel sensors 22 arranged along a plurality of rows 20r and columns 20c. A typical optical sensor 20 may support over two million pixels 22, enabling corresponding row 20r and column 20c dimensions of about 1400×1400 pixels (the so-called resolution of the optical sensor 20). To provide full-color capabilities, the optical sensor 20 must provide color information for at least three colors, which form a component color set. The most commonly used component color set is a set of the three complementary colors red, green and blue. Other sets are possible, though, such as a set of the four colors cyan, yellow, magenta and green. The optical sensor 20 is shown utilizing the more common component color set of red, green and blue. Each pixel sensor 22 is capable of providing intensity information for only one of the three colors red, green and blue. Eight bits (one byte) of intensity information is typically provided by each pixel sensor 22. Consequently, the pixel sensors 22 may be broken into interleaved sub-categories: red pixel sensors 22r, green pixels sensors 22g and blue pixel sensors 22b. The green pixel sensors 22g predominate, as the human visual system is most sensitive to the color green. Each red pixel sensor 22r provides one byte of intensity information for red. Each green pixel sensor 22g provides one byte of intensity information for green, and each blue pixel sensor 22b provides one byte of intensity information for blue. For a two mega-pixel optical sensor 20, raw image frame data will have a total size of two mega-pixels*1 byte/pixel=two megabytes. For each picture taken by a user, the image sensor system 2 thus generates two megabytes of raw image frame data.
A pre-processing unit 4 is used to provide each byte of data from a pixel sensor 22 (a so-called pixel) with color information from the other component colors, so that each pixel, after pre-processing, will have 3 bytes of color information: a byte for red intensity information, another for green, and yet another for blue 24 bits in total. The pre-processing system 4 utilizes pixel sensors 22 adjacent to a target pixel sensor 22 to calculate intensity values for the other component colors of the target pixel sensor 22. For example, consider a red pixel sensor 24r. There are eight pixel sensors 22 adjacent to the red pixel sensor 24r, which include green pixel sensors 24g and blue pixel sensors 24b. The target pixel sensor 24r physically detects and provides 8 bits of red intensity information to the pre-processing system 4. Similarly, each adjacent green pixel sensor 24g provides 8 bits of green intensity information, and each adjacent blue pixel sensor 24b provides 8 bits of blue intensity information. All of this color intensity information is used by the pre-processing unit 4 to generate an additional 16 bits of color information for the target pixel sensor 24r: 8 bits of blue, and 8 bits of green. The pre-processing system 4 thus creates full image frame data from the raw image frame data. Whereas the raw image frame data is two megabytes in size, the full image frame data coming from the pre-processing system 4 is 6 megabytes in size.
A frame buffer 10 is used to buffer and store complete image frame information. The frame buffer 10 must be large enough to hold at least one complete image frame 10f. Six megabytes of full image frame data, however, is a lot of data to handle and store. In order to reduce the memory size requires of the frame buffer 10, a high-speed compression system 6 is used to compress the full image frame data received from the pre-processing system 4. For data integrity, the high-speed compression system 6 is ideally a lossless, or near-lossless, compression system. Differential pulse coded modulation (DPCM), in conjunction with Huffman encoding, is a standard algorithm used by the high-speed compression system 6. DPCM is basically a stream of difference values derived from differences between components in a stream of data (i.e., differences in color intensity information in the data stream from the pre-processing system 4). As image data tends to be smoothly varying, these difference values require fewer bits for encoding than actual color intensity values, and tend to have a zero-sum mean, which is ideally suited for Huffman encoding. Compression ratios of 50% are generally obtained by the high-speed compression system 6. Each frame 10f within the frame buffer 10 is thus not actual full image frame data, but is instead compressed frame data derived from the full image frame data. Rather than requiring at least 6 megabytes of memory per frame 10f, the frame buffer 10, with the help of the high-speed compression system 6, can instead use but 3 megabytes of memory per frame 10f. Component costs are reduced accordingly.
A decompression system 14 is used to restore the compressed frame data in a frame 10f back into full image frame data, which is then fed into a lossy compression system 16. Lossy compression systems 16, as the name implies, tend to irrecoverably lose image information, but offer exceptional compression ratios. Such lossy compression systems 16 are designed, however, to “lose” information in such a way as to be undetectable, or nearly undetectable, by the human visual system. The JPEG compression standard is the most commonly used lossy compression algorithm for digital cameras. The lossy compression system 16 thus accepts full image frame data from the de-compression system 14 to generate JPEG image data, which is then permanently stored in non-volatile memory 18.
Getting digital cameras to market at a price that is affordable to the average consumer is of the utmost importance to manufacturers. Many tools and devices are used to bring component costs down. One such tool is the so-called system-on-a-chip (SOC) component. In effect, every integrated circuit required for the flow chart of FIG. 1 is placed onto a single silicon device, with the possible exceptions of the image sensor system 2 and the non-volatile memory 18. By far, the greatest cost associated with such an SOC design is the frame buffer 10. If the memory size requirements of the frame buffer 10 can be reduced, considerable savings may be achieved, with a correspondingly less expensive digital camera being brought to market.